Human-Robot Interaction by Understanding Upper Body Gestures

In this paper, a human-robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions and verbal language. A set of 12 upper body gestures is involved for communication. This set also includes gestures with human-object interactions. The gestures are characterized by the head, arm and hand posture information. The wearable Immersion CyberGlove II is employed to capture the hand posture. This information is combined with the head and arm posture captured from the Microsoft Kinect. This is a new sensor solution for human-gesture capture. Based on the posture data from the CyberGlove II and Kinect, an effective and real-time human gesture recognition method is proposed. The gesture understanding approach based on an innovative combination of sensors is the main contribution of this paper. To verify the effectiveness of the proposed gesture ∗Corresponding author: Institute for Media Innovation, Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798. E-mail:[email protected]